Similar item detection

ABSTRACT

A method to determine image similarities. The method may include obtaining a first image and a second image and determining a discrete transform difference between a first discrete transform of the first image and a second discrete transform of the second image. The method may also include determining multiple first intensity vectors for the first image and determining multiple second intensity vectors for the second image. The method may also include determining an intensity vector difference between the multiple first intensity vectors and the multiple second intensity vectors and determining a color difference between a first color histogram of the first image and a second color histogram of the second image. The method may also include determining a similarity between the first image and the second image based on the discrete transform difference, the intensity vector difference, and the color difference.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/586,318, filed on Dec. 30, 2014, titled “SIMILAR ITEM DETECTION”,which is incorporated herein by reference.

FIELD

The embodiments discussed herein are related to similar item detection.

BACKGROUND

Many marketplaces allow sellers to post images of merchandise that theyare selling. In some circumstances, sellers may generate multiplelistings for the same item and use the same image for each listing totry to increase the opportunities for the seller to sell the item. As aresult of the multiple listings, buyers in the marketplace searching formerchandise similar to the item may receive search results that includethe multiple listings for the same items. Returning multiple listingsfor the same item may reduce the buying experience of the buyer and/orreduce the incentives for the buyer to buy items from the marketplace.

The subject matter claimed herein is not limited to embodiments thatsolve any disadvantages or that operate only in environments such asthose described above. Rather, this background is only provided toillustrate one example technology area where some embodiments describedherein may be practiced.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments will be described and explained with additionalspecificity and detail through the use of the accompanying drawings inwhich:

FIG. 1 illustrates an example system to determine image similarities;

FIG. 2A illustrates a portion of an image similarity determinationprocess;

FIG. 2B illustrates a portion of an image similarity determinationprocess;

FIG. 2C illustrates a portion of an image similarity determinationprocess;

FIG. 2D illustrates a portion of an image similarity determinationprocess;

FIG. 3 illustrates a pixel array;

FIG. 4 illustrates an example system to determine image similarities;

FIG. 5 is a flowchart of an example method to determine imagesimilarities; and

FIGS. 6A and 6B illustrate a flowchart of another example method todetermine image similarities.

DESCRIPTION OF EMBODIMENTS

In some embodiments in an on-line marketplace, a seller of goods mayprovide multiple listings of the same goods using duplicate or nearduplicate images. In these and other embodiments, the on-linemarketplace may identify duplicate or near duplicate images bydetermining similarities in the images. In these and other embodiments,a system may be described that includes a database and a computingsystem. The database may be configured to store a first image associatedwith a first product offered for sale and a second image associated witha second product offered for sale. The computing system may becommunicatively coupled to the database.

The computing system may be configured to determine a similarity betweenthe first image and the second image and based on a result of thedetermination, alter a presentation of the second image to a buyer. Insome embodiments, the determining the similarity between the first imageand the second image may include various steps. The steps may includedetermining a discrete transform difference between a first discretetransform of the first image and a second discrete transform of thesecond image and determining multiple first intensity vectors for thefirst image. Each of the multiple first intensity vectors may correspondto a different one of multiple first pixels of the first image. Thesteps may also include determining multiple second intensity vectors forthe second image. Each of the multiple second intensity vectors maycorrespond to a different one of multiple second pixels of the secondimage. The steps may also include determining an intensity vectordifference between the multiple first intensity vectors and the multiplesecond intensity vectors and determining a color difference between afirst color histogram of the first image and a second color histogram ofthe second image. The steps may also include determining the similaritybetween the first image and the second image based on the discretetransform difference, the intensity vector difference, and the colordifference.

Alternately or additionally, some embodiments herein may describe amethod to determine image similarities. The method to determine imagesimilarities may be used by an on-line marketplace or some other systemto determine image similarities. The method may include obtaining afirst image and a second image and determining a discrete transformdifference between a first discrete transform of the first image and asecond discrete transform of the second image. The method may alsoinclude determining multiple first intensity vectors for the firstimage. Each of the multiple first intensity vectors may correspond to adifferent one of multiple first pixels of the first image.

The method may also include determining multiple second intensityvectors for the second image. Each of the multiple second intensityvectors may correspond to a different one of multiple second pixels ofthe second image. The method may also include determining an intensityvector difference between the multiple first intensity vectors and themultiple second intensity vectors and determining a color differencebetween a first color histogram of the first image and a second colorhistogram of the second image. The method may also include determining asimilarity between the first image and the second image based on thediscrete transform difference, the intensity vector difference, and thecolor difference.

Alternately or additionally, some embodiments herein may describeanother method to determine image similarities. The method to determineimage similarities may be used by an on-line marketplace or some othersystem to determine image similarities. The method may include obtaininga first image and a second image and determining multiple first valueintensity vectors for the first image. Each of the multiple first valueintensity vectors may correspond to a different one of multiple firstvalue pixels of the first image. The method may also include determiningmultiple second value intensity vectors for the second image. Each ofthe multiple second value intensity vectors may correspond to adifferent one of multiple second value pixels of the second image.

The method may also include determining a value intensity vectordifference between the multiple first value intensity vectors and themultiple second value intensity vectors and determining multiple firstset intensity vectors for the first image. Each of the multiple firstset intensity vectors may correspond to a different one of multiplefirst set pixels of the first image. The method may also includedetermining multiple second set intensity vectors for the second image.Each of the multiple second set intensity vectors may correspond to adifferent one of multiple second set pixels of the second image. Themethod may also include determining a set intensity vector differencebetween the multiple first set intensity vectors and the multiple secondset intensity vectors and determining multiple first random intensityvectors for the first image. Each of the multiple first random intensityvectors may correspond to a different one of multiple first randompixels of the first image.

The method may also include determining multiple second random intensityvectors for the second image. Each multiple of second random intensityvectors may correspond to a different one of multiple second randompixels of the second image. The method may also include determining arandom intensity vector difference between the multiple first randomintensity vectors and the multiple second random intensity vectors anddetermining a similarity between the first image and the second imagebased on the value intensity vector difference, the set intensity vectordifference, and the random intensity vector difference.

Turning to the figures, FIG. 1 illustrates an example system 100 todetermine image similarities. The system 100 may be arranged inaccordance with at least one embodiment described herein. The system 100may include a network 110, an on-line marketplace 112 that includes acomputing system 120 and a database 130, a first device 140, and asecond device 150.

The network 110 may be configured to communicatively couple the on-linemarketplace 112 with the first device 140 and the second device 150. Insome embodiments, the network 110 may be any network or configuration ofnetworks configured to send and receive communications between devices.In some embodiments, the network 110 may include a conventional typenetwork, a wired or wireless network, and may have numerous differentconfigurations. Furthermore, the network 110 may include a local areanetwork (LAN), a wide area network (WAN) (e.g., the Internet), or otherinterconnected data paths across which multiple devices and/or entitiesmay communicate. In some implementations, the network 110 may include apeer-to-peer network. The network 110 may also be coupled to or mayinclude portions of a telecommunications network for sending data in avariety of different communication protocols. In some implementations,the network 110 includes Bluetooth® communication networks or a cellularcommunications network for sending and receiving communications and/ordata including via short message service (SMS), multimedia messagingservice (MMS), hypertext transfer protocol (HTTP), direct dataconnection, wireless application protocol (WAP), e-mail, etc. Thenetwork 110 may also include a mobile data network that may includethird-generation (3G), fourth-generation (4G), long-term evolution(LTE), long-term evolution advanced (LTE-A), Voice-over-LTE (“VoLTE”) orany other mobile data network or combination of mobile data networks.Further, the network 110 may include one or more IEEE 802.11 wirelessnetworks.

The on-line marketplace 112 may be any configuration of hardware, suchas servers and databases, that are configured to provide an on-linemarketplace for sellers to sell goods to buyers. For example, theon-line marketplace 112 may be configured to provide a number ofpublishing, listing, and price-setting mechanisms whereby a seller maylist (or publish information concerning) goods or services for sale, abuyer can express interest in or indicate a desire to purchase suchgoods or services, and a price can be set for a transaction pertainingto the goods or services. The prices set for a transaction may beauction-format listing and price setting mechanisms (e.g., English,Dutch, Chinese, Double, Reverse auctions, etc.). Other types of pricelisting formats may also be used. For example, a fixed-price listingformat (e.g., the traditional classified advertisement-type listing or acatalogue listing) or a buyout-type listing format may be used. In theseand other embodiments, a buyout-type listing may be offered inconjunction with auction-format listings and allow a buyer to purchasegoods or services, which are also being offered for sale via an auction,for a fixed price that is typically higher than a starting price of anauction for an item.

As part of the listing of goods for sale, a seller may provide an imageof an item. The image of the item may provide a visual representation ofthe item to be sold. In some embodiments, a seller may create multiplelistings for the same item using the same or near duplicate images ofthe item. In some circumstances, a seller creating multiple listings forthe same item may be undesirable for buyers because multiple duplicateitems of a type of goods may be presented to the buyer when the buyersearches for the types of goods.

In some embodiments, the on-line marketplace 112 may be configured todetermine when the same or near duplicate images of an item are postedin two different listings based on a similarity between the images ofthe items. In these and other embodiments, in response to the on-linemarketplace 112 determining that the images of the item are similar, theon-line marketplace 112 may take action with respect to listings withthe same or near duplicate images of the item. In some embodiments, theaction taken by the on-line marketplace 112 may include removing alisting with the similar image or reducing the placement of the listingwith the similar image in search results provided to a buyer.

Images of items in listings may be determined to be similar images whenthe images are duplicate images. For example, a first listing mayinclude a first image and a second listing may include a second image.When the first and second images are duplicate images, the second imagemay be a digital copy of the first image. For example, a seller mayupload the same image to the on-line marketplace 112 for both the firstand second listings.

Alternately or additionally, images in a listing may be determined to besimilar images when one of the images is a modified version of anotherof the images. For example, a first listing may include a first imageand a second listing may include a second image that is a modifiedversion of the first image. To create the second image as a modifiedversion of the first image, a digital copy of the first image may becreated and manipulated through digital processing to generate thesecond image. For example, the digital copy of the first image may bedigitally manipulated by adding a watermark to the digital copy.Alternately or additionally, the digital copy may be digitallymanipulated by adjusting a color or luminance of one or more pixels ofthe digital copy. In some embodiments, the digital processingmanipulation may result in a difference between the first and secondimages that may not be perceived by a human eye, but that may bedetected by a strict comparison of the first and second images, such asa hash value comparison of the first and second images.

In some embodiments, the computing system 120 of the on-line marketplace112 may be configured to determine a similarity between the first andsecond images of different listings. In these and other embodiments, thefirst and second images of the different listings may be stored in thedatabase 130.

In some embodiments, the computing system 120 may be a single computingsystem that includes one or more processors and memory, such as a serveror some other computing system. Alternately or additionally, thecomputing system 120 may include multiple computing systems, such asmultiple servers, that are networked together and configured to performa task. The database 130 may be any configuration of digital datastorage.

To determine the similarity between the first and second images, thecomputing system 120 may obtain the images from the database 130 andobtain different features of the first and second images. For example,the computing system 120 may be configured to determine at least one ofmacro features, micro features, and color features of the first andsecond images. The computing system 120 may determine the similaritybetween the first and second images based on these features.

By using at least one of the macro features, the micro features, and thecolor features, the computing system 120 may distinguish similar images,including images that are duplicate images and images that are duplicateimages with digital manipulations. For example, using the macro featuresto consider a whole of a first and a second image, the computing system120 may determine if the first and second images are duplicate images.In some embodiments, the macro features of the first and second images,if the second image is a digitally manipulated duplicate of the firstimage, may indicate that the images are similar because the macrofeatures may be unaffected by some digital manipulations, such as watermarks. However, in some embodiments, the macro features may not properlydistinguish between the first and second images with minor differencesand thus misclassify the first and second images as similar when thefirst and second images are not similar (e.g., when the first and secondimages are not duplicates or duplicates with a digital manipulation thatwould not be visible to a human).

The micro features of the first and second images may consider smallportions of the first and second images and may properly distinguishbetween the first and second images with minor differences. The minordifferences, such as a digital manipulation of a duplicate image,between the first and second images when using the micro features mayresult in the first and second images being determined to not besimilar. By combining information from both the micro features and themacro features, a better determination of the similarity of the firstand second images may be obtained.

In some embodiments, the macro and micro features may not consider thecolors of the first and second features. For example, the macro andmicro features may use a grayscale of the first and second images. Inthese and other embodiments, the color features may also provideinformation about the similarity between the first and second images. Asmall difference in the color features may indicate that the first andsecond images are similar. In some embodiments, a large difference inthe color features along with a small difference in the macro and microfeatures may indicate that the first and second images are of a sameitem but that are different colors. Thus, the first and second imagesmay not be similar.

For example, assume that a seller lists a camera for sale in first andsecond listings using first and second images of the camera. The secondimage of the camera may be a duplicate image of the first image with asmall water mark that is not perceived in the image by viewers of thelisting. The macro features of the first and second images may indicatethat the first and second images are the same. The micro features of thefirst and second images may indicate that multiple of the micro featuresare very similar, but have minor differences. The color features mayindicate that the first and second images are similar. As a result, thecomputing system 120 may determine that the first and second images aresimilar.

As another example, assume that a seller lists a first camera for salein a first listing and a second camera for sale in a second listingusing first and second images of the first and second cameras,respectively. The first and second cameras may be cameras of the samemodel, but the first and second images may be slightly different becausethe first and second images may be taken at slightly different angles.The macro features of the first and second images may indicate that thefirst and second images are not the same. The micro features of thefirst and second images may indicate that micro features are similar butmostly not the same. The color features may indicate that the first andsecond images are very similar. As a result, the computing system 120may determine that the first and second images are not similar.

As another example, assume that a seller lists a first camera for salein a first listing and a second camera for sale in a second listingusing first and second images of the first and second cameras,respectively. The first and second cameras may be cameras of the samemodel but different colors. The grayscale macro features of the firstand second images may indicate that the first and second images are thesame. The grayscale micro features of the first and second images mayindicate that the micro features are the same. The color features mayindicate that the first and second images are not similar. As a result,the computing system 120 may determine that the first and second imagesare not similar.

In some embodiments, the computing system 120 may determine a differencebetween the macro features, the micro features, and the color featuresof the first and second images and provide the differences to asimilarity model. The similarity model may generate an indication ofwhether the first and second images are similar based on thedifferences. In some embodiments, the similarity model may be generatedusing machine learning. As a result, the similarity model may begenerated based on training images that reflect a level of similaritysuited to an operator of the on-line marketplace 112.

In some embodiments, the computing system 120 may determine the macrofeatures of the first and second images based on a discrete transform ofthe first and second images. For example, the discrete transform may bea discrete cosine transform, a discrete sine transform, or a modifieddiscrete cosine transform.

In these and other embodiments, the computing system 120 may determine afirst discrete transform for the first image and a second discretetransform of the second image. The computing system 120 may determine adiscrete transform difference between the first discrete transform andthe second discrete transform. In some embodiments, the discretetransform difference may represent a macro-level feature differencebetween the first and second images. The discrete transform differencemay be provided to the similarity model and used by the similarity modelto determine a similarity between the first and second images.

In some embodiments, the computing system 120 may determine the microfeatures of the first and second images using multiple intensity vectorsderived for each of the first and second images. In these and otherembodiments, the computing system 120 may determine multiple firstintensity vectors for the first image and multiple second intensityvectors for the second image. Each of the multiple first intensityvectors may correspond to a different pixel in the first image. Each ofthe multiple second intensity vectors may correspond to a differentpixel in the second image.

In some embodiments, the computing system 120 may determine one, two, orthree different types of multiple intensity vectors for each of thefirst and second images. One type of the multiple intensity vectors maybe value-pixel intensity vectors. Another type of the multiple intensityvectors may be set-pixel intensity vectors. Another type of the multipleintensity vectors may by random-pixel intensity vectors. For each typeof intensity vectors selected, the computing system 120 may determinemultiple intensity vectors. Each of the multiple intensity vectors foreach type of intensity vector may correspond with a different pixel inthe first and second images.

A pixel, with which a single intensity vector corresponds, may be acenter pixel used when determining the intensity vector. The intensityvector may be generated based on an intensity level and orientation ofmultiple pixels surrounding the center pixel. The intensity level of apixel may include a pixel value associated with a pixel that indicates abrightness of the pixel. An intensity orientation of a pixel may includean orientation of an intensity gradient calculated based on one of morepixels surrounding the pixel.

In general, the size of an intensity vector may be based on a level ofgranularity for grouping the intensity orientations of the pixels. Insome embodiments, a size of the intensity vector may vary for eachintensity vector type. Alternately or additionally, a number of pixelssurrounding the center pixel that are used to calculate the intensityvector may also vary for the different types of intensity vectors.

In some embodiments, the pixels to which value-pixel intensity vectorscorrespond may be pixels with illumination intensity maximums. In someembodiments, the pixels with illumination intensity maximums may bepixels at edges within an image. In some embodiments, the pixels towhich random-pixel intensity vectors correspond may be pixels selectedrandomly from an image with at least a minimum pixel distance therebetween. In some embodiments, the pixels to which set-pixel intensityvectors correspond may be pixels from a particular grid of pixels. Inthese and other embodiments, the particular grid of pixels may be pixelsthat are equal spaced in a grid throughout an image.

In some embodiments, the computing system 120 may determine multiplefirst random-pixel intensity vectors for the first image and multiplesecond random-pixel intensity vectors for the second image. Thecomputing system 120 may compare the first random-pixel intensityvectors and the second random-pixel intensity vectors to determine arandom pixel intensity difference. In some embodiments, the random pixelintensity difference may represent a micro-level feature differencebetween the first and second images. The random pixel intensitydifference may be provided to the similarity model and used by thesimilarity model to determine a similarity between the first and secondimages.

Alternately or additionally, the computing system 120 may determinemultiple first set-pixel intensity vectors for the first image andmultiple second set-pixel intensity vectors for the second image. Thecomputing system 120 may compare the first set-pixel intensity vectorsand the second set-pixel intensity vectors to determine a set-pixelintensity difference. In some embodiments, the set-pixel intensitydifference may represent a micro-level feature difference between thefirst and second images. The set-pixel intensity difference may beprovided to the similarity model and used by the similarity model todetermine a similarity between the first and second images.

Alternately or additionally, the computing system 120 may determinemultiple first value-pixel intensity vectors for the first image andmultiple second value-pixel intensity vectors for the second image. Thecomputing system 120 may compare the first value-pixel intensity vectorsand the second value-pixel intensity vectors to determine a value-pixelintensity difference. In some embodiments, the value-pixel intensitydifference may represent a micro-level feature difference between thefirst and second images. The value-pixel intensity difference may beprovided to the similarity model and used by the similarity model todetermine a similarity between the first and second images.

In some embodiments, the computing system 120 may determine the colorfeatures of the first and second images based on the saturation and huecharacteristics of the first and second images. Using the saturation andhue characteristics of the first and second images, the computing system120 may generate a first color histogram for the first image and asecond color histogram for the second image. The computing system 120may also determine a color difference between the first color histogramand the second color histogram. In some embodiments, the colordifference may represent a color level feature difference between thefirst and second images. The color difference may be provided to thesimilarity model and used by the similarity model to determine asimilarity between the first and second images.

The computing system 120 may be configured to determine similaritiesbetween images at any event that occurs with respect to the images inthe on-line marketplace 112. For example, in some embodiments, when animage is uploaded to the database 130, the computing system 120 mayaccess other images in the database and determine similarities betweenthe recently uploaded image and other images in the database.Alternately or additionally, the computing system 120 may determinesimilarities between images based on a particular schedule, such asdaily, weekly, etc., a random schedule, or a number of images uploaded,among other events.

In response to determining that images are similar, the computing system120 may remove one of the images, send a request to a seller whouploaded one of the images to remove the one of the images, or tag oneof the images such that the images do not appear together in searchresults, among other actions.

In some embodiments, the computing system 120 may determine similaritiesbetween a recently received image and all other images in the database130 or some particular set of the images in the database 130, such asimages in a similar classification of goods. For example, the computingsystem 120 may compare a recently received image from a specific sellerto other images uploaded by the specific seller. Alternately oradditionally, the computing system 120 may compare a recently receivedimage from a specific seller to other images uploaded by one or moresellers associated with the specific seller.

Alternately or additionally, the computing system 120 may determinesimilarities between images returned from a search for a good performedby a buyer. For example, a buyer may submit a search for particulargoods. The on-line marketplace 112 may conduct a search for theparticular goods and select listings for displaying to the buyer. Aftercompiling the listings for displaying to the buyer, but before sendingthe listings to the buyer over the network 110, the computing system 120may determine the similarities between images on the selected listings.In response to determining a similarity difference between images of twoof the listings, the computing system 120 and/or the on-line marketplace112 may remove one of the listings with the similar images from thelistings to display to the buyer. Alternately or additionally, inresponse to determining a similarity difference between images of two ofthe listings, the computing system 120 and/or the on-line marketplace112 may adjust the position of one of the listings with the similarimages in the listings to display to the buyer. For example, theposition of the one of the listings with the similar images may bedemoted by causing the one of the listings to be placed on a second orsubsequent page.

An example of the operation of the system 100 follows. A seller in theon-line marketplace 112 may interact with the on-line marketplace 112using the first device 140. For example, the seller may interact withthe on-line marketplace 112 using an internet browser operating on thefirst device 140. The seller may create a first listing for selling acamera and upload a first image of the camera to the on-line marketplace112 through the network 110. The on-line marketplace 112 may store thefirst image in the database 130.

The seller may also create a second listing for selling the camera andupload a second image of the camera to the on-line marketplace 112. Thesecond image may be similar to the first image, in that the second imagemay be a digital copy of the first image or a digital copy of the firstimage with processing manipulations.

In some embodiments, upon receipt of the second image, the computingsystem 120 may determine a similarity between the second image and otherimages uploaded by the seller, including the first image. In response todetermining that the first image and the second image are similar, thecomputing system 120 may remove the second listing from the on-linemarketplace 112.

Alternately or additionally, a buyer through the second device 150 mayaccess the on-line marketplace 112. The buyer may perform a search forcameras generally. The on-line marketplace 112 may search the listingsfor cameras and prepare search results that include the first and secondlistings. After preparing the search results, but before sending thesearch results to the second device 150, the computing system 120 maydetermine similarities between images of the listings in the searchresults. After determining that the first and second images are similar,the computing system 120 or the on-line marketplace 112 may remove thesecond listing from the search results or demote the second listing to alater page in the search result. The on-line marketplace 112 may thensend the search results to the second device 150 for display to thebuyer.

In some on-line marketplaces, the marketplaces may compare images ofdifferent listings to determine if the images are duplicate images andremove listings with duplicate images. To avoid having listings removed,a seller may be able to generate multiple listings using a single imagethat is digitally manipulated in multiple ways. In contrast, the on-linemarketplace 112 using the computing system 120 may determine images thatare similar and remove listings with images that are duplicate images orduplicate images with digital manipulations. As a result, the on-linemarketplace 112 may include less redundant listings and a better buyerexperience.

Generally, other marketplaces, such as print media, may not have aredundant listings problem as sellers may pay for every listing of anitem. In contrast, on-line marketplaces may charge a fee to a sellerwhen an item is sold and not when an item is listed. As a result, aseller may be motivated to include multiple listings. An on-linemarketplace, however, may desire less redundant listings for a betterbuyer experience. As noted, typical methods of detecting duplicateimages may not detect similar images when a seller digitally manipulatesa duplicate image. As a result, sellers in some on-line marketplaces maybe able to include duplicate listings. The current disclosure providesan on-line marketplace with the ability to detect similar images.Furthermore, in circumstances, the digital manipulations to duplicateimages may not be visual to a human. As a result, a human reviewing theimages may not be able to detect the similar images and reduce redundantlistings. The present disclosure, in some embodiments, describes acomputing system that may detect similar images and reduce redundantlistings to improve a user experience that may not be able to beperformed by a human.

In view of this disclosure, it will be appreciated that modifications,additions, or omissions may be made to the system 100 without departingfrom the scope of the present disclosure. For example, in someembodiments, the system 100 may not include the first and second devices140 and 150. Alternately or additionally, the on-line marketplace 112may include additional components and/or networks than those illustratedin FIG. 1.

FIG. 2A illustrates a portion of an image similarity determinationprocess 200A. The process 200A may be arranged in accordance with atleast one embodiment described herein. The process 200A may be performedusing a discrete transform module 210 (“the DT module 210”) and a firstdifference module 220.

The process 200A may include determining a discrete transform difference222 between a first image 202 and a second image 204. To begin theprocess 200A, the DT module 210 may receive the first image 202 and thesecond image 204. The DT module 210 may be configured to apply adiscrete transform to the first image 202 to generate a first transformmatrix 212. The DT module 210 may also be configured to apply a discretetransform to the second image 204 to generate a second transform matrix214. The first transform matrix 212 and the second transform matrix 214may be frequency representations of the first and second images 202 and204. In some embodiments, the discrete transform applied by the DTmodule 210 may be a discrete cosine transform, a discrete sinetransform, or a modified discrete cosine transform.

In some embodiments, the DT module 210 may process the first and secondimages 202 and 204 before applying the discrete transform to the firstand second images 202 and 204. For example, in some embodiments, the DTmodule 210 may down sample the first and second images 202 and 204 toreduce a number of pixels of the first and second images 202 and 204.For example, the DT module 210 may generate down sampled first andsecond images 202 and 204 that are 16×16, 20×20, 24×24, 32×32, or someother number of pixels. The DT module 210 may also be configured toconvert the first and second images 202 and 204 to grayscale imagesbefore applying the discrete transform.

The first and second transform matrices 212 and 214 may be provided tothe first difference module 220. The first difference module 220 may beconfigured to determine a difference between the first and secondtransform matrices 212 and 214 and to output the difference as thediscrete transform difference 222. In some embodiments, the differencemay be calculated based on a portion of the first and second transformmatrices 212 and 214. In some embodiments, the first difference module220 may compare the low frequency components of the first and secondtransform matrices 212 and 214. In these and other embodiments, thefirst difference module 220 may compare the low frequency components bycomparing a sub-matrix of the first and second transform matrices 212and 214 that includes the origin of the first and second transformmatrices 212 and 214.

In some embodiments, the first difference module 220 may encode thefirst and second transform matrices 212 and 214 into binary matrices andcompare the binary matrices to determine the discrete transformdifference. In these and other embodiments, to generate a binary matrixof the first transform matrix 212, the first difference module 220 maycalculate a mean of the values of the first transform matrix 212 andencode the values above the mean as a 1 and encode the value below orequal to the mean as zero. The first difference module 220 may generatethe binary matrix of the second transform matrix 214 in a similarmanner. In these and other embodiments, the first difference module 220may determine the discrete transform difference 222 by determining ahamming distance difference between the binary matrices.

In view of this disclosure, it will be appreciated that modifications,additions, or omissions may be made to the process 200A withoutdeparting from the scope of the present disclosure.

FIG. 2B illustrates a portion of an image similarity determinationprocess 200B. The process 200B may be arranged in accordance with atleast one embodiment described herein. The process 200B may be performedusing a color histogram module 230 and a second difference module 240.

The process 200B may include determining a color difference 242 betweenthe first image 202 and the second image 204. The first image 202 andthe second image 204 may be the same images used in the process 200Adescribed with respect to FIG. 2A.

To begin the process 200B, the color histogram module 230 may receivethe first image 202 and the second image 204. The color histogram module230 may be configured to convert the first image 202 and the secondimage 204 to a hue, saturation, value (HSV) standard. The colorhistogram module 230 may extract the first hue and saturation imageplanes of the first image 202. Using the first hue image plane, thecolor histogram module 230 may form a first hue histogram as representedby a vector. The first hue histogram may include multiple bins thatcover the spectrum of possible hue values. A magnitude of the hue valuesfrom the hue image plane may be combined in their appropriate bin in thefirst hue histogram to form the first hue histogram. Using the firstsaturation image plane, the color histogram module 230 may form a firstsaturation histogram from the first saturation plane. The combined firsthue and saturation histograms may form first color information 232 andmay be provided to the second difference module 240.

The color histogram module 230 may extract second hue and saturationimage planes of the second image 204. Using the second hue image plane,the color histogram module 230 may form a second hue histogram. Usingthe second saturation image plane, the color histogram module 230 mayform a second saturation histogram. The combined second hue andsaturation histograms may form second color information 234 and may beprovided to the second difference module 240.

The second difference module 240 may be configured to determine adifference between the first color information 232 and the second colorinformation 234. In some embodiments, to determine the differencebetween the first color information 232 and the second color information234, the second difference module 240 may compare the first huehistogram with the second hue histogram. In some embodiments, to comparethe first hue histogram with the second hue histogram the seconddifference module 240 may determine a hue Euclidean distance betweenvectors that represent the first hue histogram and the second huehistogram.

In some embodiments, to determine the difference between the first colorinformation 232 and the second color information 234, the seconddifference module 240 may compare the first saturation histogram withthe second saturation histogram. In some embodiments, to compare thefirst saturation histogram with the second saturation histogram thesecond difference module 240 may determine a saturation Euclideandistance between vectors that represent the first saturation histogramand the second saturation histogram.

The second difference module may determine the color difference 242based on a saturation Euclidean distance, the hue Euclidean distance, orsome combination of the saturation Euclidean distance and the hueEuclidean distance. For example, the color difference 242 may be a meanof the saturation Euclidean distance and the hue Euclidean distance.

In view of this disclosure, it will be appreciated that modifications,additions, or omissions may be made to the process 200B withoutdeparting from the scope of the present disclosure.

FIG. 2C illustrates a portion of an image similarity determinationprocess 200C. The process 200C may be arranged in accordance with atleast one embodiment described herein. The process 200C may be performedusing an intensity vector module 250 and a third difference module 270.The intensity vector module 250 may include a set pixel-intensity vectormodule 252 (“the set module 252”), a random pixel-intensity vectormodule 254 (“the random module 254”), and value pixel-intensity vectormodule 256 (“the value module 256”).

The process 200C may include determining multiple set pixel-intensityvectors, multiple random pixel-intensity vectors, and multiple valuepixel-intensity vectors for each of the first and second images 202 and204. The first image 202 and the second image 204 may be the same imagesused in the process 200A and process 200B described with respect toFIGS. 2A and 2B, respectively.

To begin the process 200C, the intensity vector module 250 may receivethe first image 202 and the second image 204. The intensity vectormodule 250 may be configured to determine an intensity gradient for eachof the pixels of the first and second images 202 and 204.

In some embodiments, the intensity vector module 250 may determine theintensity gradients of each of the pixels using a scale invariantfeature transform (SIFT). An example of determining the intensitygradient for pixels using SIFT follows.

In some embodiments, the intensity vector module 250 may determine theintensity gradients for each of the pixels of first and second images202 and 204 by comparing the intensity for a selected pixel with theintensities of one or more pixels surrounding the selected pixels. Forexample, to determine the intensity gradient for a first pixel, theintensity vector module 250 may compare the intensity of the first pixelwith an intensity of a second pixel above the first pixel and of a thirdpixel to the right of the first pixel. In these and other embodiments,the intensity vector module 250 may determine the intensity gradient ofthe first pixel by taking the root of the sum of the squares of theintensities differences between the first pixel and the second and thirdpixels. The intensity vector module 250 may also be configured todetermine an angle, e.g., orientation, of the intensity gradient. Theorientation of the intensity gradient of a pixel may be found based onthe following equation:

${Orientation} = {\tan^{- 1}\left( \frac{A_{1} - A_{2}}{A_{1} - A_{3}} \right)}$where A₁ is the intensity of the first pixel, A₂ is the intensity of thesecond pixel, and A₃ is the intensity of the third pixel. In someembodiments, other pixels surrounding the first pixel may be used todetermine the intensity gradient of the first pixel.

The intensity vector module 250 may provide the intensity gradients ofthe pixels of the first and second images 202 and 204 to each of the setmodule 252, the random module 254, and the value module 256.

The set module 252 may determine multiple set intensity vectors for eachof the first and second images 202 and 204 using the intensity gradientsof some of the pixels in the first and second images 202 and 204. Eachof the multiple set intensity vectors in the first image 202 maycorrespond and be calculated based on a particular pixel in the firstimage 202. Each of the multiple set intensity vectors in the secondimage 204 may correspond and be calculated based on a particular pixelin the second image 204.

To determine the multiple set intensity vectors for each of the firstand second images 202 and 204, the set module 252 may determine multipleset pixels for each of the first and second images 202 and 204. The setpixels for each of the first and second images 202 and 204 may becorresponding pixels in the first and second images 202 and 204. The setpixels may be selected based on a particular equally spaced grid appliedto the first and second images 202 and 204. For example, the grid may bea square grid that includes four points that are equally spaced apart.The four points may correspond to pixels in the first and second images202 and 204. The pixels to which the four points may correspond may bereferred to herein as the set pixels. The set pixels of the first image202 may be the first set pixels and the set pixels of the second image204 may be the second set pixels. In other embodiments, the grid mayinclude 6, 8, 9, 12, 15, 20, or some other number of points. In theseand other embodiments, the grid may be square or some other shape.

For example, FIG. 3 illustrates a pixel array 300. The pixel array 300includes 100 pixels arranged in a 10×10 grid. In an example, the gridfor selecting the set pixels may be a four point 6×6 square grid. As aresult, the set pixels may be pixels (2,2), (8,2), (2,8), and (8,8). Asanother example, the grid for selecting the set pixels may be a ninepoint 6×6 square grid. As a result, the set pixels may be pixels (2,2),(2,5), (2,8), (5,2), (5,5), (5,8), (8,2), (8,5), and (8,8).

Returning to FIG. 2C, the set module 252 may determine a set intensityvector for each of the set pixels. The set intensity vector may becalculated using the intensity gradients and orientations of the pixelssurrounding the set pixels within a region surrounding the set pixels.The size of the region surrounding each of the set pixels may vary ormay be constant. In some embodiments, the region may be a circularregion surrounding the set pixels or some other shape. In someembodiments, the regions for the set intensity vectors may be based onthe grid used to select the set pixels. In these and other embodiments,the grids and thus the regions may cover a majority of the first imageand second images 202 and 204. In some embodiments, the grids and thusthe regions may cover over 80% of the first and second images 202 and204.

In some embodiments, the set pixels may be selected as keypoints in aSIFT. The set intensity vectors may be a keypoint descriptor vector fromthe SIFT. A brief description of one example of how a keypointdescriptor is determined follows.

FIG. 3 illustrates a pixel array 300 with a set pixel 310 and a setregion 312. The pixels inside the set region 312 may be referred toherein as region pixels. The set module 252 may apply a Gaussianfunction to the intensity gradient magnitudes of each of the regionpixels, with the set pixel 310 being a center of the Gaussian function.The orientations of each of the intensity gradients of the region pixelsmay be adjusted based on the orientation of the set pixel 310. A numberof orientation groupings may be selected to cover the 360 degrees of theorientations of the intensity gradients. For example, with 4 groupings,each of the groups may cover 90 degrees. The region pixels are groupedtogether according to the orientation groupings. For example, a regionpixel with an aligned orientation of 45 degrees may be in the 0 to 90degree group and a region pixel with an aligned orientation of 150degrees may be in the 90 to 180 degree group. The Gaussian adjustedintensity gradient magnitudes of each of the region pixels in each ofthe orientation groupings may be summed. A vector representing theorientation groupings and the summed Gaussian adjusted intensitygradient magnitudes may be a set intensity vector for the set pixel 310.

Returning to FIG. 2C, the set module 252 may determine multiple firstset intensity vectors 260 for the set pixels of the first image 202 andprovide the first set intensity vectors 260 to the third differencemodule 270. The set module 252 may also determine multiple second setintensity vectors 261 for the set pixels of the second image 204 andprovide the second set intensity vectors 261 to the third differencemodule 270.

The random module 254 may determine multiple random intensity vectorsfor each of the first and second images 202 and 204 using the intensitygradients of each of the pixels. Each of the multiple random intensityvectors in the first image 202 may correspond to and be calculated basedon a particular random pixel in the first image 202. Each of themultiple random intensity vectors in the second image 204 may correspondto and be calculated based on a particular random pixel in the secondimage 204.

To determine the multiple random intensity vectors for each of the firstand second images 202 and 204, the random module 254 may determinemultiple random pixels for each of the first and second images 202 and204. The random pixels for each of the first and second images 202 and204 may be corresponding pixels in the first and second images 202 and204. The random pixels may be selected randomly from the pixels in thefirst and second images 202 and 204. In some embodiments, the randompixels may be selected randomly with a particular distance between therandom pixels. In some embodiments, the random pixels may be selectedfrom a central region of the first and second images 202 and 204. Anumber of random pixels selected may vary based on a number of pixels inthe image. In some embodiments, for a 300×300 pixel image, a number ofrandom pixels may include 5, 10, 15, 20, 25, 30, 50, 75, or 100 randompixels. A random region size (e.g., a number of pixels included in therandom region) for determining the random intensity vectors may be basedon a number of selected random pixels and a number of pixels in theimage. In some embodiments, the random region size may be smaller thanthe set region size.

Based on the random region size and the random pixels, the random module254 may determine multiple first random intensity vectors 262 for thefirst image 202 and provide the first random intensity vectors 262 tothe third difference module 270. The random module 254 may alsodetermine multiple second random intensity vectors 263 for the secondimage 204 and provide the second random intensity vectors 263 to thethird difference module 270. In these and other embodiments, the randommodule 254 may determine the first random intensity vectors 262 and thesecond random intensity vectors 263 in a manner analogous to theoperation of the set module 252 as discussed above.

The value module 256 may determine multiple value intensity vectors foreach of the first and second images 202 and 204 using the intensitygradients of each of the pixels. Each of the multiple value intensityvectors in the first image 202 may correspond and be calculated based ona particular value pixel in the first image 202. Each of the multiplevalue intensity vectors in the second image 204 may correspond and becalculated based on a particular value pixel in the second image 204.

To determine the multiple value intensity vectors for each of the firstand second images 202 and 204, the value module 256 may determinemultiple value pixels for each of the first and second images 202 and204. The value pixels may be selected based on the pixels in the firstand second images 202 and 204 with the highest intensity gradientmagnitudes. For example, the pixels from the first image 202 may besorted based on the highest intensity gradient magnitudes. A firstparticular number of the pixels in the first image 202 with the highestintensity gradient magnitudes may be selected as first value pixels. Thepixels from the second image 204 may be sorted based on the highestintensity gradient magnitudes. A second particular number of the pixelsin the second image 204 with the highest intensity gradient magnitudesmay be selected as second value pixels. The first and the secondparticular numbers may be the same or different. The first and secondvalue pixels may correspond with pixels at edges in the first and secondimages, respectively. An image edge as used herein does not refer to aphysical edge of an image. Rather, an image edge refers to a location inan image with a rapidly changing intensity level between pixels. Forexample, for an image with a bright line, the pixels at the bright linemay be edge pixels.

The first and the second particular numbers of value pixels for each ofthe first and second images 202 and 204 may vary based on a number ofpixels in the image. In some embodiments, for a 300×300 pixel image, thefirst and the second particular numbers of value pixels may include 5,10, 15, 20, 25, 30, 50, 75, or 100 pixels.

A value region size for determining the value intensity vectors may bebased on a number of value pixels and a number of pixels in the image.In some embodiments, the value region size may be smaller than the setregion size. In some embodiments, the value region size may be the sameas the random region size. In some embodiments, the value region sizefor the first value pixels may be the same or different than the valueregion size for the second value pixels.

Based on the value region size and the first value pixels, the valuemodule 256 may determine multiple first value intensity vectors 264 forthe first image 202 and provide the first value intensity vectors 264 tothe third difference module 270. Based on the value region size and thesecond value pixels, the value module 256 may also determine multiplesecond value intensity vectors 265 for the second image 204 and providethe second value intensity vectors 265 to the third difference module270. In these and other embodiments, the value module 256 may determinethe first value intensity vectors 264 and the second value intensityvectors 265 in a manner analogous to the operation of the set module 252as discussed above.

The third difference module 270 may be configured to compare the firstset intensity vectors 260 and the second set intensity vectors 261. Tocompare the first set intensity vectors 260 and the second set intensityvectors 261, the third difference module 270 may determine a Euclideandistance between corresponding vectors of the first set intensityvectors 260 and the second set intensity vectors 261. For example, afirst vector of the first set intensity vectors 260 and a second vectorof the second set intensity vectors 261 that share a corresponding setpixel in the first and second images 202 and 204 may be compared. A meanof the Euclidean distances between the corresponding pairs of the firstset intensity vectors 260 and the second set intensity vectors 261 maybe output by the third difference module 270 as the set intensitydifference 272.

The third difference module 270 may also be configured to compare thefirst random intensity vectors 262 and the second random intensityvectors 263. To compare the first random intensity vectors 262 and thesecond random intensity vectors 263, the third difference module 270 maydetermine a Euclidean distance between corresponding vectors of thefirst random intensity vectors 262 and the second random intensityvectors 263 based on the corresponding random pixels in the first andsecond images 202 and 204. A mean of the Euclidean distances between thecorresponding pairs of the first random intensity vectors 262 and thesecond random intensity vectors 263 may be output by the thirddifference module 270 as the random intensity difference 274.

The third difference module 270 may also be configured to compare thefirst value intensity vectors 264 and the second value intensity vectors265. To compare the first value intensity vectors 264 and the secondvalue intensity vectors 265, the third difference module 270 maydetermine a Euclidean distance between corresponding vectors of thefirst value intensity vectors 264 and the second value intensity vectors265. A first value intensity vector may correspond to a second valueintensity vector when the first value intensity vector is a best matchof the second value intensity vector as compared to the other secondvalue intensity vectors. A mean of the Euclidean distances between thecorresponding pairs of the first value intensity vectors 264 and thesecond value intensity vectors 265 may be output by the third differencemodule 270 as the value intensity difference 276.

In view of this disclosure, it will be appreciated that modifications,additions, or omissions may be made to the process 200C withoutdeparting from the scope of the present disclosure. For example, in someembodiments, the intensity vector module 250 may down sample the firstand second images 202 and 204 to reduce a number of pixels of the firstand second images 202 and 204. For example, the intensity vector module250 may generate down sampled first and second images 202 and 204 thatare 16×16, 20×20, 24×24, 32×32, or some other number of pixels. Theintensity vector module 250 may also be configured to convert the firstand second images 202 and 204 to grayscale images before determiningintensity gradients for each of the pixels of the first and secondimages 202 and 204. Alternately or additionally, one or more of the setmodule 252, the random module 254, or the value module 256 may not beincluded in the intensity vector module 250.

FIG. 2D illustrates a portion of an image similarity determinationprocess 200D. The process 200D may be arranged in accordance with atleast one embodiment described herein. The process 200D may include asimilarity model module 280. The similarity model module 280 may beconfigured to receive the discrete transform difference 222, the colordifference 242, the intensity difference 272, the random intensitydifference 274, and the value intensity difference 276. Based on thediscrete transform difference 222, the color difference 242, the setintensity difference 272, the random intensity difference 274, and thevalue intensity difference 276, the similarity model module 280 maydetermine a similarity between the first and second images 202 and 204.

In some embodiments, the similarity model module 280 may include asimilarity model that may receive the discrete transform difference 222,the color difference 242, the set intensity difference 272, the randomintensity difference 274, and the value intensity difference 276 asinputs. In these and other embodiments, the similarity model may be amodel generated using machine learning. In these and other embodiments,training values for the differences between images similar to those ofthe discrete transform difference 222, the color difference 242, the setintensity difference 272, the random intensity difference 274, and thevalue intensity difference 276 may be provided to train the similaritymodel to identify similar images based on the criteria of differencesprovided. Based on the training values, a creator of the similaritymodel may determine how the similarity model may determine similaritiesbetween images. In some embodiments, the machine learning technique usedto generate the similarity model may be a technique that uses decisiontrees, such as a gradient boosting machine learning technique.

Generally, the similarity model module 280 may output an indication ofthe likelihood of similarity between the first and second images 202 and204. When the likelihood of similarity is above a similarity threshold,a determination may be made that the first and second images 202 and 204are similar. When the likelihood of similarity is below the similaritythreshold, a determination may be made that the first and second images202 and 204 are not similar. The similarity threshold may be selectedbased on a tolerance for error in determining similarities between thefirst and second images 202 and 204. The tolerance for error may bebased on a system that includes the first and second images 202 and 204and goods pictured in the first and second images 202 and 204, amongother criteria. In view of this disclosure, it will be appreciated thatmodifications, additions, or omissions may be made to the process 200Dwithout departing from the scope of the present disclosure.

FIG. 4 illustrates an example system 400 to determine imagesimilarities. The system 400 may be arranged in accordance with at leastone embodiment described herein. The system 400 may include a computingsystem 410 and a database 420. The computing system 410 may include aprocessor 412, a memory 414, and data storage 416. The processor 412,the memory 414, and the data storage 416 may be communicatively coupled.

In general, the processor 412 may include any suitable special-purposeor general-purpose computer, computing entity, or processing deviceincluding various computer hardware or software modules and may beconfigured to execute instructions stored on any applicablecomputer-readable storage media. For example, the processor 412 mayinclude a microprocessor, a microcontroller, a digital signal processor(DSP), an application-specific integrated circuit (ASIC), aField-Programmable Gate Array (FPGA), or any other digital or analogcircuitry configured to interpret and/or to execute program instructionsand/or to process data. Although illustrated as a single processor inFIG. 4, it is understood that the processor 412 may include any numberof processors configured to perform individually or collectively anynumber of operations described herein. Additionally, one or more of theprocessors may be present on one or more different electronic devices,such as different servers. In some embodiments, the processor 412 mayinterpret and/or execute program instructions and/or process data storedin the memory 414, the data storage 416, or the memory 414 and the datastorage 416. In some embodiments, the processor 412 may fetch programinstructions from the data storage 416 and/or the database 420 and loadthe program instructions in the memory 414. After the programinstructions are loaded into memory 414, the processor 412 may executethe program instructions.

The memory 414 and the data storage 416 may include computer-readablestorage media for carrying or having computer-executable instructions ordata structures stored thereon. Such computer-readable storage media maybe any available media that may be accessed by a general-purpose orspecial-purpose computer, such as the processor 412. By way of example,and not limitation, such computer-readable storage media may includetangible or non-transitory computer-readable storage media includingRandom Access Memory (RAM), Read-Only Memory (ROM), ElectricallyErasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-OnlyMemory (CD-ROM) or other optical disk storage, magnetic disk storage orother magnetic storage devices, flash memory devices (e.g., solid statememory devices), or any other storage medium which may be used to carryor store desired program code in the form of computer-executableinstructions or data structures and which may be accessed by ageneral-purpose or special-purpose computer. Combinations of the abovemay also be included within the scope of computer-readable storagemedia. Computer-executable instructions may include, for example,instructions and data configured to cause the processor 412 to perform acertain operation or group of operations.

The database 420 may be communicatively coupled with the computingsystem 410. The database 420 may include a first image 402 and a secondimage 404. The database 420 may also include multiple modules, that whenexecuted by the processor 412, may cause the computing system 410 toperform operations that may determine a similarity between the first andsecond images 402 and 404.

The modules included in the database 420 may include a color histogrammodule 424, a discrete transform module 426, an intensity vector module428, a first difference module 430, a second difference module 432, athird difference module 434, and a classification model module 436. Thecolor histogram module 424, the discrete transform module 426, theintensity vector module 428, the first difference module 430, the seconddifference module 432, the third difference module 434, and theclassification model module 436 may correspond to the color histogrammodule 230, the DT module 210, the intensity vector module 250, thefirst difference module 220, the second difference module 240, the thirddifference module 270, and the similarity model module 280,respectively, of FIGS. 2A, 2B, 2C, and 2D.

In some embodiments, the processor 412 may execute the color histogrammodule 424, the discrete transform module 426, and the intensity vectormodule 428 in parallel and at the same time. Alternately oradditionally, the processor 412 may execute the color histogram module424, the discrete transform module 426, and the intensity vector module428 in serial or in some combination of parallel and serial executions.

In some embodiments, the processor 412 may execute the first differencemodule 430, the second difference module 432, and the third differencemodule 434 in parallel, serially, or in some combination of parallel andserial execution. In some embodiments, the processor 412 may execute thefirst difference module 430, the second difference module 432, and thethird difference module 434 based on when the color histogram module424, the discrete transform module 426, and the intensity vector module428 finish execution.

In some embodiments, the processor 412 may execute the classificationmodel module 436 after execution of the color histogram module 424, thediscrete transform module 426, the intensity vector module 428, thefirst difference module 430, the second difference module 432, and thethird difference module 434 and based on the outputs of the firstdifference module 430, the second difference module 432, the thirddifference module 434.

In view of this disclosure, it will be appreciated that modifications,additions, or omissions may be made to the system 400 without departingfrom the scope of the present disclosure. For example, the computingsystem 410 and the database 420 may be communicatively coupled by anetwork, such as the network 110 of FIG. 1. Alternately or additionally,the system 400 may include multiple computing systems 410 that mayoperate to execute one or more of the modules stored in the database420.

FIG. 5 is a flowchart of an example method 500 to determine imagesimilarities. The method 500 may be arranged in accordance with at leastone embodiment described herein. The method 500 may be implemented, insome embodiments, by a system, such as the system 100 and/or 400 ofFIGS. 1 and 4, respectively. Although illustrated as discrete blocks,various blocks may be divided into additional blocks, combined intofewer blocks, or eliminated, depending on the desired implementation.

The method 500 may begin at block 502, where a first image and a secondimage may be obtained.

In block 504, a discrete transform difference may be determined betweena first discrete transform of the first image and a second discretetransform of the second image.

In block 506, multiple first intensity vectors may be determined for thefirst image. Each of the multiple first intensity vectors may correspondto a different one of multiple first pixels of the first image.

In block 508, multiple second intensity vectors may be determined forthe second image. Each of the multiple second intensity vectors maycorrespond to a different one of a multiple second pixels of the secondimage.

In block 510, an intensity vector difference may be determined betweenthe multiple first intensity vectors and the multiple second intensityvectors. In some embodiments, determining the intensity vectordifference between the multiple first intensity vectors and the multiplesecond intensity vectors may include determining a preliminary intensityvector difference between corresponding vectors of the multiple firstintensity vectors and the multiple second intensity vectors andaveraging the preliminary intensity vector differences.

In block 512, a color difference may be determined between a first colorhistogram of the first image and a second color histogram of the secondimage.

In block 514, a similarity between the first image and the second imagemay be determined based on the discrete transform difference, theintensity vector difference, and the color difference. In someembodiments, the first image may be determined to be similar to thesecond image when the second image is a duplicate of the first image andwhen the second image is a duplicate of the first image with aprocessing manipulation that results in a change to the first image thatis not perceived by a human eye.

In some embodiments, determining the similarity between the first imageand the second image based on the discrete transform difference, theintensity vector difference, and the color difference may includeproviding the discrete transform difference, the intensity vectordifference, and the color difference to a similarity model generatedusing machine learning.

One skilled in the art will appreciate that, for this and otherprocesses and methods disclosed herein, the functions performed in theprocesses and methods may be implemented in differing order.Furthermore, the outlined steps and operations are only provided asexamples, and some of the steps and operations may be optional, combinedinto fewer steps and operations, or expanded into additional steps andoperations without detracting from the essence of the disclosedembodiments.

For instance, in some embodiments, the first intensity vectors may befirst first-type intensity vectors, the second intensity vectors may besecond first-type intensity vectors, the multiple first pixels may bemultiple first first-type pixels, and the multiple second pixels may bemultiple second first-type pixels. In these and other embodiments, themethod 500 may further include determining multiple first second-typeintensity vectors for the first image. Each of the multiple firstsecond-type intensity vectors may correspond to a different one ofmultiple first second-type pixels of the first image. The method 500 mayfurther include determining multiple second second-type intensityvectors for the second image. Each of the multiple second second-typeintensity vectors may correspond to a different one of multiple secondsecond-type pixels of the second image. The method 500 may furtherinclude determining a second intensity vector difference between themultiple first second-type intensity vectors and the multiple secondsecond-type intensity vectors. In these and other embodiments,determining the similarity between the first image and the second imagemay be based on the discrete transform difference, the intensity vectordifference, the second intensity vector difference, and the colordifference.

In these and other embodiments, the method 500 may further includedetermining multiple first third-type intensity vectors for the firstimage. Each of the multiple first third-type intensity vectors maycorrespond to a different one of multiple first third-type pixels of thefirst image. The method 500 may further include determining multiplesecond third-type intensity vectors for the second image. Each of themultiple second third-type intensity vectors may correspond to adifferent one of multiple second third-type pixels of the second image.The method 500 may further include determining a third intensity vectordifference between the multiple first third-type intensity vectors andmultiple second third-type intensity vectors. In these and otherembodiments, determining the similarity between the first image and thesecond image may be based on the discrete transform difference, theintensity vector difference, the second intensity vector difference, thethird intensity vector difference, and the color difference.

FIGS. 6A and 6B is a flowchart of an example method 600 to determineimage similarities. The method 600 may be arranged in accordance with atleast one embodiment described herein. The method 600 may beimplemented, in some embodiments, by a system, such as the system 100and/or 400 of FIGS. 1 and 4, respectively. Although illustrated asdiscrete blocks, various blocks may be divided into additional blocks,combined into fewer blocks, or eliminated, depending on the desiredimplementation.

The method 600 may begin at block 602, where a first image and a secondimage may be obtained.

In block 604, multiple first value intensity vectors may be determinedfor the first image. Each of the multiple first value intensity vectorsmay correspond to a different one of multiple first value pixels of thefirst image.

In block 606, multiple second value intensity vectors may be determinedfor the second image. Each of the multiple second value intensityvectors may correspond to a different one of multiple second valuepixels of the second image. In some embodiments, the multiple firstvalue pixels of the first image may be selected based on illuminationintensity maximums in the first image. Alternately or additionally, themultiple second value pixels of the second image may be selected basedon illumination intensity maximums in the second image.

In block 608, a value intensity vector difference may be determinedbetween the multiple first value intensity vectors and the multiplesecond value intensity vectors.

In block 610, multiple first set intensity vectors may be determined forthe first image. Each of the multiple first set intensity vectors maycorrespond to a different one of multiple first set pixels of the firstimage. In some embodiments, determining the multiple first set intensityvectors for one of multiple first set intensity vectors may includecombining illumination intensities of multiple set region pixels thatsurround a set pixel of the multiple first set pixels that correspondswith the one of the first set intensity vectors.

In block 612, multiple second set intensity vectors may be determinedfor the second image. Each of the multiple second set intensity vectorsmay correspond to a different one of multiple second set pixels of thesecond image.

In some embodiments, the multiple first set pixels of the first imageand the multiple second set pixels of the second image may becorresponding pixels in the first image and the second image that arebased on a particular grid of pixels. In these and other embodiments,the particular grid of pixels may be an equal spaced grid of pixelsdistributed through the first and second images.

In block 614, a set intensity vector difference may be determinedbetween the multiple first set intensity vectors and the multiple secondset intensity vectors.

In block 616, multiple first random intensity vectors may be determinedfor the first image. Each of the multiple first random intensity vectorsmay correspond to a different one of multiple first random pixels of thefirst image. In some embodiments, determining the multiple first randomintensity vectors for one of the multiple first random intensity vectorsmay include combining illumination intensities of multiple random regionpixels that surround a random pixel of the multiple random pixels thatcorresponds with the one of the first random intensity vectors. In someembodiments, the multiple set region pixels may include more pixels thanthe multiple random region pixels.

In block 618, multiple second random intensity vectors may be determinedfor the second image. Each of the multiple second random intensityvectors may correspond to a different one of multiple second randompixels of the second image.

In some embodiments, the multiple first random pixels of the first imageand the multiple second random pixels of the second image may becorresponding pixels in the first image and the second image that areselected randomly. In these and other embodiments, the multiple firstrandom pixels and the multiple second random pixels may be selectedrandomly with at least a minimum distance there between.

In block 620, a random intensity vector difference may be determinedbetween the multiple first random intensity vectors and the multiplesecond random intensity vectors.

In block 622, a similarity between the first image and the second imagemay be determined based on the value intensity vector difference, theset intensity vector difference, and the random intensity vectordifference.

In some embodiments, the method 600 may further include determining adiscrete transform difference between a first discrete transform of thefirst image and a second discrete transform of the second image. Themethod 600 may also include determining a color difference between afirst color histogram of the first image and a second color histogram ofthe second image. In these and other embodiments, determining thesimilarity between the first image and the second image may be based onthe discrete transform difference, the value vector difference, the setintensity vector difference, the random intensity vector difference, andthe color difference.

As indicated above, the embodiments described herein may include the useof a special purpose or general purpose computer (e.g., the processor412 of FIG. 4) including various computer hardware or software modules,as discussed in greater detail below. Further, as indicated above,embodiments described herein may be implemented using computer-readablemedia (e.g., the memory 414 of FIG. 4) for carrying or havingcomputer-executable instructions or data structures stored thereon.

In some embodiments, the different components, modules, engines, andservices described herein may be implemented as objects or processesthat execute on a computing system (e.g., as separate threads). Whilesome of the system and methods described herein are generally describedas being implemented in software (stored on and/or executed by generalpurpose hardware), specific hardware implementations or a combination ofsoftware and specific hardware implementations are also possible andcontemplated.

Terms used herein and especially in the appended claims (e.g., bodies ofthe appended claims) are generally intended as “open” terms (e.g., theterm “including” should be interpreted as “including, but not limitedto,” the term “having” should be interpreted as “having at least,” theterm “includes” should be interpreted as “includes, but is not limitedto,” etc.).

Additionally, if a specific number of an introduced claim recitation isintended, such an intent will be explicitly recited in the claim, and inthe absence of such recitation no such intent is present. For example,as an aid to understanding, the following appended claims may containusage of the introductory phrases “at least one” and “one or more” tointroduce claim recitations. However, the use of such phrases should notbe construed to imply that the introduction of a claim recitation by theindefinite articles “a” or “an” limits any particular claim containingsuch introduced claim recitation to embodiments containing only one suchrecitation, even when the same claim includes the introductory phrases“one or more” or “at least one” and indefinite articles such as “a” or“an” (e.g., “a” and/or “an” should be interpreted to mean “at least one”or “one or more”); the same holds true for the use of definite articlesused to introduce claim recitations.

In addition, even if a specific number of an introduced claim recitationis explicitly recited, those skilled in the art will recognize that suchrecitation should be interpreted to mean at least the recited number(e.g., the bare recitation of “two recitations,” without othermodifiers, means at least two recitations, or two or more recitations).Furthermore, in those instances where a convention analogous to “atleast one of A, B, and C, etc.” or “one or more of A, B, and C, etc.” isused, in general such a construction is intended to include A alone, Balone, C alone, A and B together, A and C together, B and C together, orA, B, and C together, etc. For example, the use of the term “and/or” isintended to be construed in this manner.

Further, any disjunctive word or phrase presenting two or morealternative terms, whether in the description, claims, or drawings,should be understood to contemplate the possibilities of including oneof the terms, either of the terms, or both terms. For example, thephrase “A or B” should be understood to include the possibilities of “A”or “B” or “A and B.”

However, the use of such phrases should not be construed to imply thatthe introduction of a claim recitation by the indefinite articles “a” or“an” limits any particular claim containing such introduced claimrecitation to embodiments containing only one such recitation, even whenthe same claim includes the introductory phrases “one or more” or “atleast one” and indefinite articles such as “a” or “an” (e.g., “a” and/or“an” should be interpreted to mean “at least one” or “one or more”); thesame holds true for the use of definite articles used to introduce claimrecitations.

Additionally, the use of the terms “first,” “second,” “third,” etc. arenot necessarily used herein to connote a specific order. Generally, theterms “first,” “second,” “third,” etc., are used to distinguish betweendifferent elements. Absence a showing of a specific that the terms“first,” “second,” “third,” etc. connote a specific order, these termsshould not be understood to connote a specific order.

All examples and conditional language recited herein are intended forpedagogical objects to aid the reader in understanding the invention andthe concepts contributed by the inventor to furthering the art, and areto be construed as being without limitation to such specifically recitedexamples and conditions. Although embodiments of the present disclosurehave been described in detail, it should be understood that the variouschanges, substitutions, and alterations could be made hereto withoutdeparting from the spirit and scope of the present disclosure.

What is claimed is:
 1. A computer-implemented method, comprising:obtaining a first image from a database, the first image associated witha first database entry; obtaining a second image from a database, thesecond image associated with a second database entry; determining, usinga computing system, a similarity between the first image and the secondimage; and in response to determining that the first image is similar tothe second image, altering the second database entry in the database,altering the second database entry in the database includes one or moreof: removing the second image from the database, altering the secondimage, and changing a status in the second database entry regardingplacement of information from the second database entry in results froma search request, wherein determining the similarity between the firstimage and the second image includes: determining a plurality of firstset intensity vectors for the first image, each of the plurality offirst set intensity vectors corresponding to a different one of aplurality of first set pixels of the first image; determining aplurality of second set intensity vectors for the second image, each ofthe plurality of second set intensity vectors corresponding to adifferent one of a plurality of second set pixels of the second image,the plurality of the second set pixels corresponding to the plurality ofthe first set pixels; determining a plurality of first random intensityvectors for the first image, each of the plurality of first randomintensity vectors corresponding to a different one of a plurality offirst random pixels of the first image that are selected randomly in thefirst image; determining a plurality of second random intensity vectorsfor the second image, each of the plurality of second random intensityvectors corresponding to a different one of a plurality of second randompixels of the second image, the plurality of second random pixelsselected based on and corresponding to the plurality of first randompixels; determining a set intensity vector difference between only theplurality of first set intensity vectors and the plurality of second setintensity vectors; and determining a random intensity vector differencebetween only the plurality of first random intensity vectors and theplurality of second random intensity vectors, wherein the similaritybetween the first image and the second image is determined based on theset intensity vector difference and the random intensity vectordifference.
 2. The computer-implemented method of claim 1, wherein thefirst image is determined to be similar to the second image when thesecond image is a duplicate of the first image and when the second imageis a duplicate of the first image with a digital processing manipulationthat results in a change to the second image that is not perceived by ahuman eye.
 3. The computer-implemented method of claim 1, wherein thesimilarity between the first image and the second image is determinedwhen the first and second images are selected for presentation inresults from a search of entries in the database or in response to boththe first image and second image being stored in the database.
 4. Thecomputer-implemented method of claim 1, wherein the similarity betweenthe first image and the second image is determined in response to thefirst image and second image being obtained by the database from a sameuser.
 5. The computer-implemented method of claim 1, wherein determiningthe similarity between the first image and the second image furtherincludes: determining a plurality of first value intensity vectors forthe first image, each of the plurality of first value intensity vectorscorresponding to a different one of a plurality of first value pixels ofthe first image; determining a plurality of second value intensityvectors for the second image, each of the plurality of second valueintensity vectors corresponding to a different one of a plurality ofsecond value pixels of the second image; and determining a valueintensity vector difference between the plurality of first valueintensity vectors and the plurality of second value intensity vectors,wherein the similarity between the first image and the second image isdetermined based on the set intensity vector difference, the valueintensity vector difference, and the random intensity vector difference.6. The computer-implemented method of claim 5, wherein determining thesimilarity between the first image and the second image furtherincludes: applying a discrete transform to the first image to generate afirst transform matrix that is a frequency representation of the firstimage; applying the discrete transform to the second image to generate asecond transform matrix that is a frequency representation of the secondimage; and determining a discrete transform difference between the firsttransform matrix of the first image and the second transform matrix ofthe second image, wherein the similarity between the first image and thesecond image is determined based on the set intensity vector difference,the value intensity vector difference, the discrete transformdifference, and the random intensity vector difference.
 7. Thecomputer-implemented method of claim 6, wherein determining thesimilarity between the first image and the second image furtherincludes: generate first color information of the first image based onone or more first color histograms of hue and saturation of the firstimage; generate second color information of the second image based onone or more second color histograms of hue and saturation of the secondimage; and determining a color difference between the first image andthe second image using the first color information and the second colorinformation, wherein the similarity between the first image and thesecond image is determined based on the set intensity vector difference,the value intensity vector difference, the color difference, thediscrete transform difference, and the random intensity vectordifference.
 8. One or more non-transitory computer-readable media havinginstructions stored thereon that, responsive to execution by one or morecomputing systems, cause the one or more computing systems to performthe computer-implemented method of claim
 1. 9. A computer-implementedmethod, comprising: obtaining a first image from a database, the firstimage associated with a first database entry; obtaining a second imagefrom a database, the second image associated with a second databaseentry; determining, using a computing system, a similarity between thefirst image and the second image; and in response to determining thatthe first image is similar to the second image, altering the seconddatabase entry in the database, altering the second database entry inthe database includes one or more of: removing the second image from thedatabase, altering the second image, and changing a status in the seconddatabase entry regarding placement of information from the seconddatabase entry in results from a search request, wherein determining thesimilarity between the first image and the second image includes:applying a discrete transform to the first image to generate a firsttransform matrix that is a frequency representation of the first image;applying the discrete transform to the second image to generate a secondtransform matrix that is a frequency representation of the second image;determining a plurality of first random intensity vectors for the firstimage, each of the plurality of first random intensity vectorscorresponding to a different one of a plurality of first random pixelsof the first image that are selected randomly in the first image; anddetermining a plurality of second random intensity vectors for thesecond image, each of the plurality of second random intensity vectorscorresponding to a different one of a plurality of second random pixelsof the second image, the plurality of second random pixels selectedbased on and corresponding to the plurality of first random pixels,wherein the similarity between the first image and the second image isdetermined based on the first transform matrix, the second transformmatrix, the plurality of first random intensity vectors, and theplurality of second random intensity vectors.
 10. Thecomputer-implemented method of claim 9, wherein determining thesimilarity between the first image and the second image furtherincludes: determining a plurality of first set intensity vectors for thefirst image, each of the plurality of first set intensity vectorscorresponding to a different one of a plurality of first set pixels ofthe first image, the plurality of first set pixels selected based on ageometric shape; determining a plurality of second set intensity vectorsfor the second image, each of the plurality of second set intensityvectors corresponding to a different one of a plurality of second setpixels of the second image, the plurality of the second set pixelsselected based on and corresponding to the plurality of the first setpixels; and determining a set intensity vector difference between theplurality of first set intensity vectors and the plurality of second setintensity vectors, wherein the similarity between the first image andthe second image is further determined based on the set intensity vectordifference.
 11. The computer-implemented method of claim 10, whereindetermining the similarity between the first image and the second imagefurther includes: determining a plurality of first value intensityvectors for the first image, each of the plurality of first valueintensity vectors corresponding to a different one of a plurality offirst value pixels of the first image, each of the plurality of firstvalue pixels including intensity gradient magnitudes higher thanintensity gradient magnitudes of pixels of the first image that are notpart of the plurality of first value pixels and the intensity gradientmagnitudes of each of the plurality of first value pixels determinedbased on an intensity value of the each of the plurality of first valuepixels and one or more pixels adjacent to the each of the plurality offirst value pixels; determining a plurality of second value intensityvectors for the second image, each of the plurality of second valueintensity vectors corresponding to a different one of a plurality ofsecond value pixels of the second image, each of the plurality of secondvalue pixels including intensity gradient magnitudes higher thanintensity gradient magnitudes of pixels of the second image that are notpart of the plurality of second value pixels and the intensity gradientmagnitudes of each of the plurality of second value pixels determinedbased on an intensity value of the each of the plurality of second valuepixels and one or more pixels adjacent to the each of the plurality ofsecond value pixels; and determining a value intensity vector differencebetween the plurality of first value intensity vectors and the pluralityof second value intensity vectors, wherein the similarity between thefirst image and the second image is further determined based on thevalue intensity vector difference, and the set intensity vectordifference.
 12. The computer-implemented method of claim 11, whereindetermining the similarity between the first image and the second imagefurther includes: generate first color information of the first imagebased on one or more first color histograms of hue and saturation of thefirst image; generate second color information of the second image basedon one or more second color histograms of hue and saturation of thesecond image; and determining a color difference between the first imageand the second image using the first color information and the secondcolor information, wherein the similarity between the first image andthe second image is further determined based on the set intensity vectordifference, the value intensity vector difference, and the colordifference.
 13. The computer-implemented method of claim 9, whereindetermining the similarity between the first image and the second imagefurther includes: determining a random intensity vector differencebetween the plurality of first random intensity vectors and theplurality of second random intensity vectors; and determining a discretetransform difference between the first transform matrix of the firstimage and the second transform matrix of the second image, wherein thesimilarity between the first image and the second image is determinedbased on the discrete transform difference and the random intensityvector difference.
 14. The computer-implemented method of claim 13,wherein determining the similarity between the first image and thesecond image further includes: determining a plurality of first valueintensity vectors for the first image, each of the plurality of firstvalue intensity vectors corresponding to a different one of a pluralityof first value pixels of the first image, each of the plurality of firstvalue pixels including intensity gradient magnitudes higher thanintensity gradient magnitudes of pixels of the first image that are notpart of the plurality of first value pixels and the intensity gradientmagnitudes of each of the plurality of first value pixels determinedbased on an intensity value of the each of the plurality of first valuepixels and one or more pixels adjacent to the each of the plurality offirst value pixels; determining a plurality of second value intensityvectors for the second image, each of the plurality of second valueintensity vectors corresponding to a different one of a plurality ofsecond value pixels of the second image, each of the plurality of secondvalue pixels including intensity gradient magnitudes higher thanintensity gradient magnitudes of pixels of the second image that are notpart of the plurality of second value pixels and the intensity gradientmagnitudes of each of the plurality of second value pixels determinedbased on an intensity value of the each of the plurality of second valuepixels and one or more pixels adjacent to the each of the plurality ofsecond value pixels; and determining a value intensity vector differencebetween the plurality of first value intensity vectors and the pluralityof second value intensity vectors, wherein the similarity between thefirst image and the second image is determined based on the randomintensity vector difference, the value intensity vector difference, andthe discrete transform difference.
 15. The computer-implemented methodof claim 9, wherein determining the similarity between the first imageand the second image further includes: determining a plurality of firstvalue intensity vectors for the first image, each of the plurality offirst value intensity vectors corresponding to a different one of aplurality of first value pixels of the first image, each of theplurality of first value pixels including intensity gradient magnitudeshigher than intensity gradient magnitudes of pixels of the first imagethat are not part of the plurality of first value pixels and theintensity gradient magnitudes of each of the plurality of first valuepixels determined based on an intensity value of the each of theplurality of first value pixels and one or more pixels adjacent to theeach of the plurality of first value pixels; determining a plurality ofsecond value intensity vectors for the second image, each of theplurality of second value intensity vectors corresponding to a differentone of a plurality of second value pixels of the second image, each ofthe plurality of second value pixels including intensity gradientmagnitudes higher than intensity gradient magnitudes of pixels of thesecond image that are not part of the plurality of second value pixelsand the intensity gradient magnitudes of each of the plurality of secondvalue pixels determined based on an intensity value of the each of theplurality of second value pixels and one or more pixels adjacent to theeach of the plurality of second value pixels; and determining a valueintensity vector difference between the plurality of first valueintensity vectors and the plurality of second value intensity vectors,wherein the similarity between the first image and the second image isfurther determined based on the value intensity vector difference. 16.One or more non-transitory computer-readable media having instructionsstored thereon that, responsive to execution by one or more computingsystems, cause the one or more computing systems to perform thecomputer-implemented method of claim
 9. 17. The computer-implementedmethod of claim 9, wherein the first image is determined to be similarto the second image when the second image is a duplicate of the firstimage and when the second image is a duplicate of the first image with adigital processing manipulation that results in a change to the secondimage that is not perceived by a human eye.
 18. The computer-implementedmethod of claim 9, wherein the similarity between the first image andthe second image is determined when the first and second images areselected for presentation in results from a search of entries in thedatabase or in response to both the first image and second image beingstored in the database.
 19. The computer-implemented method of claim 9,wherein the similarity between the first image and the second image isdetermined in response to the first image and second image beingobtained by the database from a same user.